Repository navigation
DEV-1146: take an unsolicited status push off the receive buffer, in any state - #24
Merged
Merged
Conversation
…any state The firmware pushes [0xFF] 0x8A 0x71 <status> [crc] whenever its state changes for a reason the host did not cause: a dock or undock, the user button, a trial-duration expiry or a low-battery stop (log-and-stream-common docs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md §5.4). startProcessing() knew nothing of it: - Idle, it acted only on a leading 0xFF and dispatched on commandSent. A prefixed push was read as the waiting command's reply or left at the head of the buffer, and an unprefixed one was never consumed. Every later reply queued behind it, and with no command timeouts every later call hung. - Streaming, it sliced PacketSize+1 bytes without looking at the 0x00 header, so one push shifted every packet after it. Every pass now asks Shimmer3StatusPush.frame first, in any state and whatever command is waiting. A push comes off whole, with or without its ACK prefix and with its session CRC. Its status goes to deviceStatus and to a new delegate method, shimmerProtocolNewDeviceStatus, which has a default so existing delegates compile unchanged. - Width: one status byte, or two on a Shimmer3R from LogAndStream v1.00.024 (§5.3). Before connect() has read the versions, the byte after the first decides, and with a CRC on, the CRC does. - A push saying streaming stopped, once the stream has delivered a packet, returns the state to CONNECTED; the firmware sends no packet after it. So sendStopStreamingCommand now returns at once when not streaming, because only the streaming branch reads a stop's ACK. - The CRC moved to ShimmerUtilities so the framing and the tests share one copy; Shimmer3Protocol.shimmerUartCrcCalc calls it. Shimmer3DeviceStatusTest covers the framing, sizing and decoding, and ShimmerUtilitiesTest checks the CRC against the firmware's Python reference. The receive loop itself has no test: Shimmer3Protocol needs a live CBPeripheral. Unbuilt and untested: written without Xcode, so CI is the first compile. Co-Authored-By: Claude Opus 5.5 <[email protected]>
The one conflict is at the head of the receive loop, where #18's NACK check and this branch's status-push check both sit ahead of the state branches. They are chained as if push / else if NACK / else if STREAMING; the order does not matter, because a NACK starts 0xFE and a push 0xFF or 0x8A. Co-Authored-By: Claude Opus 5.5 <[email protected]>
JongChern
approved these changes
Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DEV-1146: every Shimmer3 and Shimmer3R firmware pushes its status unasked whenever its state changes for a reason the host did not cause. This API had no handler for that push, so it hung every later command, or misaligned the stream.
What the firmware sends
[0xFF] 0x8A 0x71 <status…> [crc…], on a dock or undock, a user-button press, a trial-duration expiry or a low-battery stop. The host's own starts and stops do not push. The protocol is in log-and-stream-commondocs/SHIMMER3_BT_COMMUNICATION_PROTOCOL.md, §3.2, §5.3 and §5.4.SET_INSTREAM_RESPONSE_ACK_PREFIX_STATE(0xA3) turns it off, and the firmware turns it back on at every disconnect.ShimBt_instreamStatusRespSend).§5.4 sets two rules for a host:
What was happening
startProcessing()broke both rules:receivedBytes.first == 0xFF, then dispatched oncommandSent.awaithung.PacketSize+1bytes without checking the 0x00 header, so one push misaligned every packet after it.What this changes
Shimmer3StatusPush.framelooks there before either branch reads the buffer.statusPayloadBytes).connect()has read them, the byte after the first status byte decides, because a second status byte is only ever 0x00 or 0x01.deviceStatusand to a new delegate method,shimmerProtocolNewDeviceStatus(message:). The method has a default implementation, so the example apps' delegates compile unchanged. A push whose CRC fails is consumed, to keep framing, but not acted on.CONNECTED, because the firmware sends no packet after it (ShimSens_saveDatasends only whilebtStreamingis set). Two exceptions:TASK_STARTSENSINGsetsbtStreaming, so a push built in that gap says "stopped" and the stream starts anyway.sendStopStreamingCommandnow returnstruewhen not streaming. Only the streaming branch reads a stop's ACK, so a stop sent after the device stopped by itself would have blocked every later reply. Both example apps send one from a button without checking state. The check runs after the existing in-flight wait, so a Stop pressed during a Start still goes through.ShimmerUtilities.shimmerUartCrcCalc, so the framing and the tests share one copy.Shimmer3Protocol.shimmerUartCrcCalcnow calls it, and the algorithm is unchanged.Tests
Shimmer3StatusPushandShimmer3DeviceStatuslive in the newShimmer3DeviceStatus.swift, becauseShimmer3Protocolcannot be constructed in a test.New
Shimmer3DeviceStatusTest:usbPluggedIn;statusPayloadBytesacross hardware, firmware identifier and version;New in
ShimmerUtilitiesTest: the CRC against vectors from the firmware's ownExtras/python_scripts/Shimmer_common/shimmer_crc.py. Four framing tests also use those vectors, so neither depends on this API's CRC agreeing with itself.Each new file has its four pbxproj entries.
Not tested: the wiring in
Shimmer3Protocol. That covers the call instartProcessing(), the state change, the delegate call and the stop guard. The class needs a liveCBPeripheral.Verification
Unbuilt and untested. This was written on a machine with no Swift toolchain, so this PR's CI run is the first compile and the first time any of it executes. A green check alone does not prove the tests ran. Confirm with
gh run view <id> --log: look for** TEST SUCCEEDED **and apassedline for eachShimmer3DeviceStatusTestcase.The framing algorithm was prototyped in Python against the firmware's reference CRC before being written in Swift:
It had no misreads in any case a device can send. With no CRC, a one-byte push directly followed by 0x00 or 0x01 reads as two bytes. That cannot happen before the versions are known, because no stream is running then and no reply starts with either byte. That run is evidence for the algorithm, not for the Swift.
Hardware-verify:
CONNECTED.Not fixed here
getInfoMemandgetCalibDumpCommandhandlers wait 0.5 s, then take the whole buffer as their reply. A push that lands in that window fails the reply's CRC, andconnect()then force-unwraps the nil result. Several other handlers end withreceivedBytes.removeAll()and drop a push queued behind their reply. Both behaviours predate this change.Merging with the open work
Checked with
git merge-treeagainst each branch:if … else if …chain; the order does not matter, because a NACK starts 0xFE and a push 0xFF or 0x8A.FW_IDENTIFIER_LOGANDSTREAM(Shimmer3Protocol's andShimmer3StatusPush's), so whichever lands second can point one at the other.Sibling APIs
The same push handling was audited in the other host APIs: Java and C# at master, pyshimmer at main.
ShimmerBluetooth.java)clearSerialBuffer()). Mid-stream, a push not directly after a data packet is lost. The push CRC is read but never checked. The width is right.ShimmerBluetooth.cs,ShimmerLogAndStream.cs)_ack_queue.get_nowait()raises on an empty queue. It disables the prefix by default from LogAndStream 0.15.4, but is exposed before that, and before the 0xA3 is ACKed. The width is fixed in open seemoo-lab/pyshimmer#49. It has no Bluetooth CRC.Shimmer3RClienthandles the push. DEV-1142 coversShimmer3Client.🤖 Generated with Claude Code